This pull request has merge conflicts. Could you please resolve them @0xsamalt? 🙏

Thank you for this PR @0xsamalt!

Deterministic timings in the test scenario would be great. Still, I don't think we should switch the Delay implementation between test and non-test configuration. In the end we want to test the code that is run in production, which uses futures_timer::Delay.

I haven't see the flaky kad test in the wild for a while; did you recently ran into issues with it?

Hey @elenaf9 ,

That completely makes sense! I agree that testing the exact production dependencies is generally much safer.

To answer your question: yes, I did recently run into this flakiness locally. My CPU had a tiny lag spike during the test runner, so a timer that was supposed to hit the assertion at < 10ms overshot to 12ms and panicked.

I attempted to introduce Tokio's start_paused Virtual Time here under the assumption that Kademlia was trying to move aggressively towards deterministic test execution, but I completely understand wanting to keep futures_timer::Delay intact across all environments!

Since the flakiness only seems to pop up on heavily loaded/laggy local machines, how would you like to proceed?

1. Option A: I can happily close this PR so we don't add unnecessary mock objects just to appease a rare edge case.
2. Option B: If you'd like to protect against the lag edge-case without swapping the timer, I can rip out the Virtual Time code in this PR and simply widen the time assertion bounds (e.g. subtracting 10ms for lag allowance) on those specific lines so they don't panic on slow runners.

Let me know what you prefer!